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I.  INTRODUCTION 


The  goal  of  this  research  effort  has  been  to  address  one  of  the  many 
questions  that  must  be  answered  to  determine  If  optical  Instruments  are 
feasible  based  on  the  multiaperture  optics  principle. 

Although  the  scope  t  •'  this  effort  la  modest ,  a  very  Important  concept  Is 
explored.  This  concept  involves  the  recognition  of  extended  objects  In  a 
coarse  grid  of  pixels  where  the  total  number  of  pixels  Is  small  (<100).  With 
the  constraints  Imposed  by  this  system,  any  object  has  to  be  described  as 
consisting  of  squares  and/or  rectangles  defined  by  the  pixels  located  on  an 
evenly  spaced  grid.  Objects  whose  edges  do  not  line  up  parallel  or  normal  to 
the  pixel  grid  will  appear  distorted  and  unrecognizable.  In  order  to  mitigate 
this  problem,  either  the  sensing  element  must  be  rotated  until  alignment  of 
the  pixel  grid  and  the  object's  edges  Is  achieved  or  a  rotation  has  to  be 
achieved  through  software  manipulation  of  the  data.  While  movement  of  the 
sensing  element  Is  simple  and  straightforward,  it  is  not  always  practical  or 
possible.  Therefore,  we  will  explore  the  viability  of  software  rotation  of 
the  object  to  achieve  alignment  and  recognition.  Recognition  of  objects  can 
be  achieved  by  assignment  of  binary  numbers  to  the  squares  and  rectangles 
which  make  up  the  object  and  by  comparison  with  known  numbers.  Further 
discussion  of  this  concept  is  carried  out  in  the  next  section. 

It  is  the  goal  of  this  research  effort  to  examine  just  one  of  the  many 
problems  and  constraints  imposed  by  a  multiaperture  optical  system  employing 
microprocessor  based  detection  and  recognition  hardware  and  software,  namely  a 
low  pixel  number  pattern  recognition  scheme  with  software  generated  rotation 
ability. 

Multiaperture  Optics,  the  electro-mechanical  analogon  of  the  insect  eye, 
offer  a  large  field  of  view  albeit  under  reduced  resolving  power*. 


The  biggest  advantage  of  vision  systems  based  on  multiaperture  optics  is 
their  small  physical  size  and  the  possibility  to  perform  the  computation  neces¬ 


sary  for  image  evaluation  and  pattern  recognition  in  the  eye  itself. 

The  consequence  of  this  is  that  the  object  to  be  recognized  will  be 
described  by  only  a  few  pixels.  Specialization  of  a  system  to  the  case  of  low 
pixel  number  pattern  recognition  allows  reducing  of  the  necessary  computations 
to  a  manageable  level  so  that,  indeed,  using  VLSI  technology,  image  processing 
in  the  eye  Itself  becomes  possible. 

A  pattern  recognition  scheme  using  only  rectangles  and  squares  as  ele- 
ments  of  the  picture  language  was  described  in  an  earlier  paper  .  The  scheme 
assumes  that  all  shapes  can  be  broken  down  into  rectangles  or  squares.  If  the 
rectangle  extends  in  the  x-direction  it  is  considered  an  x-feature  and  given 
the  code  01,  a  y-feature  is  given  10,  and  a  square  would  be  an  x-y  feature 
with  the  code  11;  all  this,  regardless  of  the  size  of  the  rectangles.  The 
sequence  of  rectangles  which  makes  up  the  image,  therefore,  is  represented  by 
a  recognition  number  which  is  a  sequence  containing  the  appropriate  codes  as 
defined  above.  This  recognition  number  is  stored  in  memory.  If  an  unknown 
object  is  encountered,  its  recognition  number  is  computed  with  a  simple 
program.  Then  the  obtained  number  is  looked  up  in  the  memory  and  the  shape  so 
recognized.  Therefore,  this  system  will  only  find  things  it  is  supposed  to 
find  and  will  have  no  problem  ignoring  anything  else.  The  amount  of  data 
collected  is  optically  limited  instead  of  electronically  sorted  out.  A 
special  problem,  which  is  posed  by  using  low  pixel  numbers,  is  created  by  the 
straight  lines  of  the  object.  Since  the  picture  language  consists  only  of 
rectangles  and  squares,  the  straight  lines  are  very  important  for  f  -nation  of 
the  language  elements.  Of  course,  if  the  straight  lines  of  an  object  are 
misaligned  with  the  detector  grid,  distortions  occur.  If  low  pixel  numbers 


are  used  for  the  description  of  the  object,  these  distortions  become  so  severe 
that  recognition  of  the  object  becomes  impossible. 

For  this  reason,  it  is  necessary  to  rotate  any  image  obtained,  through 
various  sets  of  degrees  (e.g.  15s,  30°  and  45°)  of  a  quadrant  until  a  recog¬ 
nizable  image  of  the  unknown  object  is  obtained. 

This  could  be  accomplished,  by  rotating  the  eye  mechanically.  As  a  matter 
of  fact,  indications  are  that  insects,  which  also  use  low  pixel  numbers  and 
therefore  have  necessarily  the  same  problems,  actually  do  this  by  aligning  the 
horizontal  band  on  their  eyes  with  the  horizon  or  their  vertical  bands  with 
vertical  edges,  e.g.,  trees.  For  a  mechanical  device  physical  rotation  of  the 
eye  is  not  very  practical,  therefore,  the  rotation  of  the  image  inside  the  eye 
is  preferable. 

This  means  for  each  image  obtained,  one  would  ask  the  question,  how  this 
image  of  an  unknown  object  would  look  if  it  had  been  obtained  with  the  object 
having  a  different  degree  of  orientation  in  respect  to  the  detector  grid? 

There  will  be  one  position  where  the  object  becomes  recognizable  and  this  is 
when  at  least  some  of  the  straight  lines  of  the  object  are  lined  up  with  the 
detector  grid. 

The  rotation  of  the  image  can  be  accomplished  either  by  hard  wiring  or  by 
sof tware . 

To  explain  how  rotation  by  hard  wiring  could  be  accomplished,  Figure  1 
shows  how  a  multiaperture  optics  system  may  be  configured.  The  optical 
elements  may  be  nonimaging  concentrators  (light  horns),  the  field  of  view 
(FOV)  of  which  may  be  controlled  by  a  field  lens.  The  FOV  of  each  '  >cet  is 
observed  by  one  fiber  shaped  detector,  so  that  the  contents  of  the  FOV  of  an 
Individual  facet  is  represented  by  one  pixel  only.  The  FOV's  of  neighboring 
facets  are  adjacent  and  do  not  overlap  significantly.  An  object,  if  large 


Figure  1  Multiaperture  optics  configuration 


enough,  is  seen  by  n  facets,  and  is  therefore  represented  by  n  pixels.  Below 
the  layer  containing  the  detectors,  a  layer  of  memory  cells  is  located  into 
which  the  information  the  detectors  received  is  loaded,  maybe  after  some 
thresholding  and  clipping  was  performed.  The  sum  of  the  memory  cells  make  up 
a  computer  core.  Therefore  the  image  resides  now  as  a  binary  image  in  a 
computer  core  having  an  organization  identical  to  the  detector  grid.  A  rota¬ 
tion  can  now  be  accomplished  by  either  software  or  hard  wiring.  Reference  2 
treats  a  45°  rotation  using  a  matrix.  As  it  turns  out  there  is  no  straight¬ 
forward  correspondence  between  an  origin  pixel  and  a  destination  pixel.  It  1 
necessary  to  divide  origin  as  well  as  destination  pixels  into  several  sub¬ 
pixels  and  have  rules  as  to  how  many  subpixels  in  a  destination  pixel  have  to 
be  occupied  in  order  to  consider  the  destination  pixel  occupied.  The  rules 
are  involved  and  therefore  require  additional  computing  capacity.  Such  a 
rotation  is  best  done  by  software. 

In  contrast  to  this  (in  this  paper)  the  possibility  of  performing  rota¬ 
tions  by  hard  wiring  is  explored.  It  is,  in  principle,  possible  to  locate 
three  more  computer  cores  underneath  the  one  discussed  above  which  could  be 
physically  rotated  in  respect  to  each  other.  Or  if  not  physically  rotated, 
still  hard  wired  connections,  connecting  an  address  in  one  core  to  a  corres¬ 
ponding  address  in  the  next  core,  could  exist.  The  corresponding  addresses 
may  be,  e.g.,  for  15°,  30°  and  45°  rotation.  It  is  still  true  that  there  is 
no  straightforward,  one-to-one  correspondence  between  locations.  However,  as 
will  be  shown  in  this  paper,  only  a  very  simple  software  procedure  will  be 
necessary  to  obtain  a  satisfactory  image  after  hard  wired  rotation  has  been 
performed. 
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II.  HARD-WIRED  ROTATION 

A.  General  Remarks 

In  the  following  a  grid  of  15  x  15  pixels  Is  assumed  to  be  only  a  part  of 
the  overall  field  of  view,  yet  large  enough  to  contain  one  object,  which  is 
supposed  to  be  recognized.  It  was  felt  that  15  x  15  is  the  smallest  size  to 
be  useful.  Larger  fields,  of  course,  experience  less  distortion,  therefore, 
it  is  necessary  to  show  that  the  smallest  practical  field  size  can  be  handled 
by  the  proposed  rotation  scheme.  It  was  also  assumed  that  a-priori  knowledge 
exists  as  to  what  objects  can  be  expected  to  be  encountered.  Only  then  is  the 
size  15  x  15  of  use. 

B.  Rotation  by  45° 

Figure  2  shows  a  15  x  15  grid  and  the  numbering  system  used.  The  number 
before  the  comma  refers  to  the  row  while  the  number  after  the  comma  refers  to 
the  column.  The  45°  rotation  is  accomplished  by  connecting  certain  pixels 
(r,c)  (r:row,  crcolumn)  in  one  memory,  the  origin  memory  "A"  to  certain  pixrls 
to  another  memory,  the  destination  memory  ”B.”  The  pixel  (8,8)  is  the  center 
pixel  of  the  rotation,  therefore  A(8.8)  is  connected  to  B(8.8). 

To  find  the  necessary  other  connections  it  has  to  be  realized  that  the 
image  is  distorted  to  begin  with.  If  the  object  was  indeed  observed  in  a  45° 
position,  then  the  direction  which  would  be  the  x-direction  on  the  object,  if 
it  were  lined  up  with  the  grid,. is  observed  too  long  by  a  factor  of  /JT  For 
this  reason,  the  obtained  (rotated)  dimension  needs  to  be  shortened  by  a 
factor  of  77T.  Since  one  deals  with  a  binary  image  one  can  do  this  only  by 
occasionally  removing  or  moving  certain  pixels,  at  a  strategic  location. 

For  ease  of  illustrations  the  connections  between  the  memories  "A”  and 
”B"  are  made  in  steps.  Step  No.  1:  The  main  diagonal  line  A(l,l)  through 
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A(15,15)  is  connected  to  center  column  of  B,  which  is  column  8,  as  indicated 
in  Figure  2.  Next  the  neighboring  diagonals  A(l,2)  through  A(14,15)  are 
connected  to  the  neighboring  rows,  in  a  way  that  A(l,2)  connects  to  B(2^9)  and 
A<1,3)  to  BO,  10),  etc. 

To  illustrate  the  consequences  of  the  different  steps  of  such  connections 
an  example  is  given  in  Figure  3.  The  object  is  depicted  in  Figure  3a,  while 
Figure  3b  shows  how  it  is  observed  by  a  15  x  15  multiaperture  optics  system  if 
the  straight  edges  of  the  object  are  inclined  by  45°  to  the  detector  grid. 

The  result  of  the  step  1  connections  can  be  seen  in  Figure  3c.  As  can  be 
seen,  the  image  is  still  highly  distorted,  and  some  modifications  of  the 
connections  have  to  be  made.  The  next  step  therefore  should  be  to  correct  for 
the  /2~distortion  introduced  by  the  diagonals  of  the  square  pixels. 

Step  No.  2:  Shift  Columns  B(ll)  through  B(15)  and  B(l)  through  B(5)  up 
by  one  unit  and  remove  all  pixels  from  columns  B(>13)  and  B(<3). 

The  result  of  step  2  is  depicted  in  Figure  3d.  As  can  be  seen  by  com¬ 
paring  Figure  3d  to  Figure  3a,  the  image  is  too  short  in  the  y-direction. 

This  problem  is  solved  in  the  next  step. 

Step  No.  3:  Insert  an  empty  row  between  row  10  and  row  11,  in  a  way  that 
the  old  row  11  becomes  row  12.  Also  move  the  contents  of  the  pixel  blocks 
B(12  through  15,  5  and  6)  as  well  as  B(12  through  15,  10  and  11)  one  unit 
toward  the  center.  If  two  occupied  pixels  should  fall  on  top  of  each  other, 
the  affected  pixel  is  considered  just  as  one  occupied  pixel.  Figure  3a  shows 
the  result  of  step  no.  3. 

This  concludes  the  hard  wired  rotation.  However  the  obtained  result 
still  needs  some  improvement.  This  has  to  be  done  by  software  since  the 
actions  to  be  taken  depend  on  the  shape  of  the  observed  object.  The  required 
procedure  however  is  very  simple.  The  program  goes  through  all  rows  and 


inserts  a  binary  Ml“  Into  all  pixels  containing  a  "0"  and  having  a  neighbor 
containing  a  “1"  on  each  side.  The  same  is  done  for  columns.  The  result  of 
this  procedure  is  shown  in  Figure  3f. 

As  can  be  seen,  the  obtained  image  corresponds  well  to  the  original,  so 
well  that  the  recognition  routine  of  Reference  1  will  recognize  it. 

Figure  4  shows  the  45°  rotation  of  two  more  shapes.  Parts  (a)  and  (aa) 
of  Figure  4  shows  the  original  shapes  while  parts  (b)  and  (bb)  show  them  as 
seen  by  the  system  if  some  of  the  straight  edges  are  inclined  by  45°  in 
respect  to  the  grid.  The  part  (c)  and  (cc)  finally  show  the  result  of  the 
rotation.  Again  the  results  are  very  close  to  the  original. 

In  general  it  can  be  said  that  if  the  object  is  lying  in  a  45°  position, 
most  of  the  object  pixels  are,  in  fact,  a  part  of  the  45°  axis  or  at  least  of 
some  line  parallel  to  it.  This  fact  favors  the  hard  wire  rotation.  In  the 
other  cases,  where  the  object  is  lying  in  a  position  different  from  45°,  the 

object  pixels  no  longer  form  a  part  of  a  relative  axis.  Here  a  software 
3  A  5 

rotation-1  *  ’  or  a  software  controlled  hard  wire  rotation  is  required.  Again 
we  approach  the  destination  grid  in  a  number  of  sequential  steps. 

Rotation  by  30°: 

Figure  5(a)  is  the  object  lying  at  0°  position.  Figure  5(b)  is  the  same 
object  viewed  as  if  lying  at  a  30s  position.  We  notice  the  object  shape  is 
less  distorted  as  compared  to  the  previous  case  (45#),  so  the  x-direction 
reduction  factor  has  to  be  smaller.  The  object  size  is  actually  shortened  by 
a  factor  of  2/  /7*ln  the  x-directlon.  Again,  we  assume  the  pixel  (8,8)  as 
the  center  pixel  or  rotation. 

STEP  1:  RIGHT  SHIFT  the  pixels  of  rows  10  through  15  by  two  units  and  row  9 


by  one  unit. 

UP  SHIFT  the  contents  of  columns  10  through  15  by  two  units  and 
columns  9  by  one  unit. 


:  DOWN  SHIFT  Che  contents  1  through  6  by  one  unit. 

:  LEFT  SHIFT  the  contents  of  rows  1  through  4  by  two  units  and  row  5  by 
one  unit.  DOWN  SHIFT  the  contents  of  rows  I  through  3  by  one  unit. 
The  result  of  this  sequential  shifting  aligns  the  object  vertical  lines 
to  the  grid  lines  but  gives  a  distorted  picture  due  to  undesirable  pixels  at 
certain  locations. 

STEP  2:  Remove  pixels  in  block  size  2  x  1  on  both  sides  of  center  pixel.  The 
result  is  a  less  distorted  picture. 

STEP  3:  This  is  the  same  kind  of  improvement  as  performed  in  the  45°  rotation 
case.  The  program  goes  through  all  rows  and  inserts  a  binary  "1" 
into  all  object  pixels  containing  a  'O'  and  having  a  neighbor 
containing  a  1 1*  on  each  side.  The  same  is  done  for  columns.  The 
result  of  this  procedure  is  Figure  5(c). 

Rotation  by  15°; 

Again,  we  assume  the  pixel  (8,8)  is  the  center  pixel  of  rotation.  The 
reduction  factor  in  this  case  is  /2~ (V^J-l)  so  we  have  to  shorten  the  object  in 
x-direction  by  this  factor.  Figure  6(b)  is  the  object  viewed  as  if  it  were 
lying  at  15°  position. 

STEP  1:  SHIFT: 

LEFT  SHIFT  the  contents  of  rows  1  through  5  by  one  unit. 

UP  SHIFT  the  contents  of  columns  9  through  15  by  one  unit. 

RIGHT  SHIFT  the  contents  of  rows  10  through  15  by  one  unit. 

UP  SHIFT  the  contents  of  blocks  (rows  9  through  15  and  columns  8 
through  15)  and  block  (row  1  through  9  and  column  11  through  15)  by 
one  unit. 

STEP  2:  Remove  pixels  in  block  size  2  x  2  on  each  vertical  side  of  the  center 


pixel.  By  doing  this,  most  of  the  distortion  is  reduced 


STEP  3:  Again,  for  improvement  the  program  goes  through  all  rows  and  inserts 
a  binary  "1"  into  all  object  pixels  containing  a  *0'  and  having  a 
neighbor  containing  a  '1*  on  each  side.  The  same  is  done  for  the 
columns . 

The  result  of  this  whole  procedure  is  Figure  6(c). 

III.  CONCLUSIONS 

The  described  multiaperture  optics  system  deals  with  low  pixel  numbers. 
This  allows  handling  of  the  collected  information  with  a  minimum  of  compu¬ 
tation.  The  use  of  low  pixel  numbers  results  in  distortion  of  straight  lines 
of  the  object.  Since  these  lines  would  no  longer  be  parallel  with  the  detec¬ 
tor  grid,  they  appear  distorted  and  recognition  of  the  object  becomes  diffi¬ 
cult  or  impossible.  It  was  shown  in  this  paper  that  this  problem  can  be 
solved  by  rotation  of  the  observed  image  before  recognition  is  attempted. 
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